Augmented, virtual and mixed-reality content selection &amp; display

ABSTRACT

Systems, methods and techniques for automatically recognizing two or 3-dimensional real world objects with an augmented reality display device (smartphone or glasses etc.), and augmenting or enhancing the display of such real world objects by superimposing virtual images such as a still or video advertisement, an opportunity to buy, a story or other virtual image presentation. In non-limiting embodiments, the real world object includes visible features including visible security features and a recognition process takes the visible security features into account when recognizing the object and/or displaying superimposed virtual images.

CROSS-REFERENCES TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 17/228,814,filed Apr. 13, 2021, now US Patent No. ______; which is a continuationof U.S. patent application Ser. No. 16/893,236 filed Jun. 4, 2020, nowU.S. Pat. No. 10,997,418; which is a continuation-in-part of U.S. patentapplication Ser. No. 16/565,234 filed Sep. 9, 2019, now U.S. Pat. No.10,699,124; and said U.S. patent application Ser. No. 16/893,236 filedJun. 4, 2020 also claims benefit of U.S. Provisional Patent ApplicationNo. 62/966,503, filed Jan. 27, 2020. Each of these prior applications isincorporated herein by reference for all purposes.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

None.

BACKGROUND & SUMMARY

Augmented reality (“AR”) is the real-time superimposition of artificialimages onto real world images. This technology has been used for avariety of applications from handheld entertainment to heads-up displayson military jets. For example, a person using an augmentedreality-enabled display device (e.g., a smartphone with an AR app or ARenabled glasses) while shopping can be shown virtual advertisements,sale announcements, pricing information etc. superimposed onto images ofactual products they can purchase.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is an example process flow diagram.

FIG. 1B shows an example branching process.

FIG. 1C shows an example area of recognition.

FIGS. 2A-2J show example items to be recognized.

FIG. 3 shows an example block diagram of a smart device and associatedsystem.

FIG. 4 shows an example non-limiting process flow diagram.

FIGS. 5A and 5B show an example augmented reality image superimposing avirtual image onto image captures.

FIGS. 6A and 6B show an example augmented reality image superimposing avirtual image (a video) onto an image capture of the front cover of a USpassport.

FIG. 7A shows an example capture of an image of a print advertisement ona smart device.

FIG. 7B shows an example area of recognition of the FIG. 7A imagecapture.

FIG. 7C shows an example augmented reality image including superimposedvirtual images (e.g., virtual shopping carts) onto the FIG. 7B imagecapture.

FIG. 8A shows an example image capture of a print advertisement.

FIG. 9A shows an example area of recognition of the FIG. 8A imagecapture.

FIG. 9B shows an example augmented reality image superimposing virtualimages (virtual shopping carts) onto the FIG. 8A image capture.

FIG. 10A shows an example image capture of a product trade dressincluding an area of recognition.

FIG. 10B shows an example augmented reality image superimposing avirtual image (e.g., a video of ingredients) onto the image capture ofthe FIG. 10A product trade dress.

FIG. 11A shows an example image capture of a real estate printadvertisement including an area of recognition.

FIG. 11B shows a closeup view of the FIG. 11A area of recognition.

FIG. 11C shows an example augmented reality image superimposing avirtual image (e.g., a listing of additional information concerning theproperty) onto the image capture of FIG. 11A.

FIG. 12 shows an example employment listing.

DETAILED DESCRIPTION OF EXAMPLE NON-LIMITING EMBODIMENTS

Because AR tends to drive higher levels of attention, one advantageoususe of augmented reality is superimposing advertisements and/orpurchasing information over product images. People who might ignoretraditional electronic advertisements may pay attention to an ARadvertisement. Same with traditional printed advertisements, which canbe used as a trigger to superimpose a digital overlay which then opensup new options like adding videos, live streams, buying options or otherinformation and content.

This technology provides systems, methods and techniques forautomatically recognizing two- or three dimensional real world objects(any kind of print or the package of a product) with an augmentedreality display device, and augmenting or enhancing the display of suchreal world objects by superimposing virtual images such as a still orvideo advertisement, a story, a tutorial, specific information or othervirtual image presentation.

In non-limiting embodiments, the real world object includes visiblefeatures including visible security features and a recognition processtakes the visible security features into account when recognizing theobject and/or displaying superimposed virtual images.

In the near future, when AR becomes really popular with any kind ofglasses or other viewing arrangement, we might also see at least apartial replacement of the smartphone. That means that we will see newuse cases that use AR or mixed reality to provide users or customerswith new information, any kind of content or buying options triggered bythe recognition of a print that could be a 2 or 3 dimensional object.For example, if you see a job advertisement, with your AR glasses or ahandheld device you can watch a related video or activate a call buttonto talk to somebody. When you see a travel advertisement, you point thecamera or other viewing device on this and you can watch a video of thedestination, a buy option or different dates etc. When you see anadvertisement of a product, you can decide to watch a related video, getother product information or buy it. This same scenario can exist with aproduct or service manual that could be the trigger of specificinformation that will allow you to understand how a product works bygiving you technical drawings etc. This same scenario could apply in thereal estate market where you can see a video of the object, specificinformation such as a neighborhood, floorplan, a call button to call theagent etc. This same scenario could apply with tickets where there is acity light as a trigger and when the AR kicks in, you can buy thisticket for a concert, a sport event etc. It could be that specificinformation about that concert or event can be provided to motivatecustomers to purchase. That could be videos or other specificinformation about players, a track record, etc. In one exampleembodiment, a device enables or allows superimposition of a digitaloverlay that leads to specific action such as providing specificinformation; providing videos, tutorials, or any kind of content;providing options to buy; or other options.

FIG. 1A shows an example non-limiting process. At 10, a printed,engraved, stamped or other object 102 such as a banknote, coin or ID(see FIGS. 2A-2J) is scanned to produce a digital copy. Such scanningcan be performed with a camera, a CMOS light sensor, a document scanner,or any other conventional device that creates a digital image of theprinted object.

The printed object may include visible features such as distinctiveimages of persons, buildings, etc. In the example shown, the printedobject further includes an example visible security feature Srepresented here as a vertical band. The scanned digital copy willinclude the distinctive image(s) as well as the visible security featureS.

At 20, the resulting digital copy of the printed object is uploaded intoa database, and enhanced to provide a selected associated overlaycomprising: (a) self-starting video(s) and/or video(s) that start whenyou physically press a button that has been programmed to show up on thesurface of a device like a mobile phone, Ipad etc., or activate thisactivity by voice, movements of your head and/or gesture; (b) button(s)that when selected have different interactive functions such as “tell astory”; and (c) interactive connection to an online shopping experiencee.g., to show a store finder (see We can change the digital “overlay”quickly if needed (e.g., to change from one advertising campaign toanother) or update content, price tags, videos or other information.

To activate that overlay, we determine an “area of recognition” asindicated by a cross-hatched area (30). In one example embodiment, thearea of recognition is determined at least in part by the presence,position, dimensions and/or orientation of one or more security featuresS. For example, the area of recognition may be defined to exclude thesecurity feature S on the printed object. As an example, in the case ofa 20 Euro note (see FIG. 2E) including a vertically-oriented silverholographic band which appears differently depending on the viewingangle, the “area of recognition” may defined to exclude the silverholographic band so the system is able to recognize the object evenwithout a silver line. Similarly, the area of recognition in the case ofa $100 US banknote shown in FIG. 2C may be defined to exclude securityfeatures B, C and E since at least some of these features may appeardifferently depending on viewing angle (e.g., the ink color changes fromgold to green as viewing angle changes). Other security features such asserial numbers, bar codes etc. vary from one printed document toanother, so that using such indicia will result in no recognition of abanknote of the same denomination but having a different unique serialnumber and/or unique bar code (i.e., the purpose of the recognition maybe to recognize a particular kind of banknote, not that specificone-of-a-kind banknote). Excluding such security features from the areaof recognition makes recognition more robust because it is not subjectto such constraints.

The recognition of a specific area then triggers a digital overlay thatincludes defined features (video, information, content, shopping, etc.)that show up and can be used by a person to get information, see avideo, shop etc.

In another embodiment at 40, at least some security features S areincluded in the defined area of recognition. The object is recognizedonly if it includes the security feature(s). If there are visiblesecurity features, we can include them into our recognition program.When we upload the digital copy of a print, we can decide which area ofthe print is used for recognition. See FIG. 1C. When this area ofrecognition includes an example such as the silver line of a banknote,the note cannot be recognized when the silver line is missing. In thisembodiment, the security feature S is used as a feature of recognitionto increase the reliability of the recognition to recognize distinctivedocuments, while designing the recognition algorithm to take differentviewing angles into account. For example, it is possible to include theposition and orientation of a silver holographic band in an area ofrecognition without requiring the holographic contents of the band to berecognized. Similarly, it is possible to include the position of serialnumbers or bar codes in an area of recognition without requiring thespecific serial numbers or bar code contents to match.

Some embodiments provide plural overlapping areas of recognition for thesame object; one area of recognition may exclude certain securityfeatures and another area of recognition includes those securityfeatures. The plural different overlapping areas of recognition can beapplied sequentially or simultaneously to increase recognitionreliability. A voting algorithm can be used to select positive matches.

In example non-limiting embodiments, the database enables real timerecognition of an image captured by a user. For example, if the usercaptures an image of a 20 Euro note, a matching algorithm is used todetermine a positive match if the database contains a digital copy of a20 Euro note. In example non-limiting embodiments, the matchingalgorithm can include pattern recognition techniques such as describedin Conti et al, Multiresolution image registration, Proceedings.,International Conference on Image Processing (IEEE 23-26 Oct. 1995);Hasanuzzaman et al, Robust and effective component-based banknoterecognition by SURF features, 20th Annual Wireless and OpticalCommunications Conference (IEEE 15-16 Apr. 2011); Doush et al, Currencyrecognition using a smartphone: Comparison between color SIFT and grayscale SIFT algorithms, Journal of King Saud University—Computer andInformation Sciences Volume 29, Issue 4, October 2017, Pages 484-492.

As will be understood by those skilled in the art, the database 20 couldbut need not contain the captured images themselves. For example, insome embodiments, the database might contain compressed or other featuresets used for comparing with captured photos, such feature sets forexample comprising a listing of coordinates and associated imagefeatures to thereby reduce storage requirements and increase recognitionspeed. Similarly, when a smart device captures an image, instead ofuploading the entire image it may analyze the image and upload acompressed format such as coordinates of pattern features. Since thepurpose of recognition in the example non-limiting embodiment is not todetermine whether the banknote or other printed item is authentic andgenuine, the matching/recognition standards can be significantly relaxedand thus quite different as compared to conventional banknotescanners/recognizers.

Some example embodiments use artificial intelligence and machinelearning to perform the matching. The training set consists of imagescaptured by various smartphones and other user devices. Otherembodiments use simpler pattern recognition algorithms based on colorand/or grey scale information in the area(s) of recognition. Any givenobject can have at least one and as many as N different discrete areasof recognition, where N is a positive integer.

FIG. 3 shows an example non-limiting system 200 including a user devicesuch as a smartphone 202, AR glasses, etc. which includes a camera orCMOS sensor 204, a processor 206, a memory 208, a display 210 and userinput/output devices 212 such as a touch screen, push buttons, etc. Theprocessor 206 communicates wirelessly or through a wire via a network214 with a server 216 which may store the database 20 and in someembodiments perform the image recognition. The app may compriseexecutable instructions that the device 202 downloads from the server216, stores in memory 206, and executes in response to user selection ofinput/output devices 212.

From a user perspective, as shown in FIG. 1A, after downloading an apponto their smart phone, smart glasses or other portable or non-portabledevice 202, the customer can point the device's camera onto a print,physical object or a digital copy on the screen of his tablet, computer,etc. (100; FIG. 4 blocks 302, 304, 306). This can be the camera ofa_device like an Iphone or an external camera or a camera that is a partof AR glasses. This causes the device 202 to display a photographicallycaptured image on display 210 (see FIG. 5A, 6A). It is possible in someembodiments for device 202 to obtain this captured image some other waythan photographing an actual object 102, such as via network 214, e.g.,by downloading the image through Instagram or other photo sharingservice, by receiving the image via a text (SMS) message, by receivingthe image from another device in proximity (e.g., using Bluetoothpairing, etc.) or by other means.

Once the app has been activated to recognize the captured image, the appconnects with the database 20 on server 216 via network 214 and checksif this print is recognized (FIG. 4 blocks 308, 310). If recognition issuccessful (branch 110), an action is enabled (e.g., playing a video,telling a story and/or showing a picture, showing shoplinks or otherinformation) (FIG. 4 , blocks 312, 314). If recognition is unsuccessful(branch to 120), no action is enabled or a different action (“xyz”) maybe performed instead.

In one non-limiting example, the app causes the smartphone or otherdevice 202 to show a picture or video by anchoring it to the imagecurrently being captured by the device's camera 204 and displayed on thedevice's display 210. See FIG. 5B, 6B. If the user moves the deviceslightly to change the position or orientation of the printed item onthe device's's display 210, the picture or video will move with theimage of the printed item on the display as if it were glued orotherwise adhered to the printed item or were being displayed by theprinted item itself from a magic display window on the printed item. Forexample, the superimposed image can change position, orientation and/orperspective as the underlying real world object changes position,orientation and/or viewing angle in order to provide a photorealisticimage in which the superimposed image appear to be part of theunderlying real world image. See for example FIG. 6B. This augmentedreality feature provides a sense of wonder or surprise that is designedto capture the attention of the user. Instead of appearing to be a“flat” advertisement, the picture or window appears to be much moreinteractive and interesting and thus captures the user's attention. Theaugmented reality image can be animated and can be accompanied by audioand/or tactile sensations as indicated in FIGS. 5A, 5B, 6A, 6B.

In other embodiments, so-called “mixed reality” or “virtual reality” canbe used instead of or in addition to augmented reality. Generallyspeaking, “virtual reality” presents the user with an image of anentirely virtual/non-real world. The posture (orientation and/orposition) of the device may be sensed (e.g., using inertial sensors suchas accelerometers and/or gyrosensors; magnetic position and/ororientation sensors; RF beacon sensors; optical sensors providing anoptical “flow” image analysis; and other known conventionaltechnologies. Different portions of the virtual world may be displayedbased on such sensed posture.

Other embodiments can use so-called “mixed reality” (MR) that mixesvirtual and real scenes. Such an MR experience is one where the user isplaced in an interactive setting that is either real with virtual assetaugmentation (augmented reality), or virtual with real-worldaugmentation. See Ohta et al, Mixed Reality: Merging Real and VirtualWorlds (Springer Publishing 2014); Milgram et al, A Taxonomy of MixedReality Visual Displays, IEICE TRANSACTIONS on Information and SystemsVol.E77-D No. 12 pp. 1321-1329 (1994/12/25); Hughes et al, Mixed realityin education, entertainment, and training, IEEE Computer Graphics andApplications (Volume: 25, Issue: 6, November-December 2005).

In more detail, a user can aim the camera of a smartphone, smart glassesor other electronic device at any two-dimensional or three-dimensionalreal world object or a likeness of such an object. One example type of atwo-dimensional object could be a portable, wallet-sized planar objectsuch as a banknote, driver's license, passport, printed advertisement,real estate advertisement or other official or unofficial identificationdocument examples of which are shown in FIG. 2A-2I. Other examples ofsuch objects might include for example a real estate advertisement, ahuman resources want ad, and objects related to a shopping experience.Using the technology with AR Glasses instead of a mobile phone means,that you do not touch a button of the superimposed digital image on yourscreen but press either a virtual button to shop or get information orjust say that you want to press a specific button or say that you wantto see a video or say that you want to buy a merchandise that is offeredwithin the digital overlay,

For example, the object as shown in FIG. 7 may comprise a real estateadvertisement which can show one or several houses, lots, commercialbuildings, private properties etc. An example area of recognition mayconstitute a single advertisement or a portion of an advertisement (seeFIG. 8 ). Image recognition may be used to recognize the particular realestate property, and such recognition may be used to select and initiatedisplay of a media item that provides more images and informationconcerning that property.

These information can be floor plans, videos of the property, contactdata of the agent, the lot size, drone videos, the actual price tag orany other information that ate relevant for the seller, the buyer or theagent.

The same with any shopping or travel advertisements where you define anarea of recognition which could be a part or parts of that advertisementor even the full page to trigger a superimposed image that containsvideos, shoplinks or other buttons that provide other content orinformation and allow to buy merchandise, travel related services(flight, hotel, tour, cruise, travel package, insurance and/or watch,read related information.

Another example for a use case is related to packaging, where you definean area of recognition on a 3 dimensional object. After a match in thedatabase you can add product information, videos, or any other contentor buy the specific merchandise.

As another example, the object shown in FIG. 9 is an employmentadvertisement. Image recognition processes may be used to recognize theadvertisement. Successful recognition can be used to select and causedisplay of a media item that provides more information and imagesconcerning the employment opportunity. For example, there is a stronguse case where you point the camera on a standard HR advertisement andthen you see, for example, the CEO or the HR Manager of the companydescribing that specific job.

As yet another example, the object may comprise the face, head or bodyof a person or animal. Image recognition processes may be used torecognize the person or animal. Successful recognition can be used toselect and cause display of a media item that provides more informationand images concerning the recognized person or animal.

Yet another example uses an object comprising a shopping item such as aparticular product or service offering. Image recognition processes maybe used to recognize the product or service offering. Successfulrecognition can be used to select and cause display of a media item thatprovides more information and images concerning the recognized productor service offering, or buying options

In one example, such objects may be made of paper, plastic and/or othermaterials, and may be rigid, flexible and/or foldable. They may be ofstandard sizes such as 2.61 inches wide and 6.14 inches long; 120 by 62millimetres (4.7 in×2.4 in) to 160 by 82 millimetres (6.3 in×3.2 in);the ID-1 format size of 85.60×53.98 mm (3⅜ in×2⅛ in approx.) and roundedcorners with a radius of 2.88-3.48 mm (about ⅛ in); ID-3 standard sizeof 125×88 mm (4.921×3.465 in); or other size. See for example ISO/IEC7810:2003. The object may be two-dimensional or three-dimensional.

One example such object comprises a banknote such as a US dollar bill,US five dollar bill, US ten dollar bill, US twenty dollar bill (see FIG.2A), US fifty dollar bill (see FIG. 2B), a US one-hundred dollar bill(see FIG. 2C), a 20 euro note (see FIG. 2D), a 50 euro note (see FIG.2E), or any other currency banknote. Other example objects compriseofficial government-issued identification such as a government issuedpassport (see FIG. 2F), passport card (see FIG. 2G), driver's license(see FIG. 2H, 2I), coin (FIG. 2J) or other official or unofficialobject.

Such two-dimensional objects as described above often are protected byany or all of the following visible security features:

holograms such as portrait window(s)

woven three-dimensional security ribbon(s)

distinctive image pattern(s) such as the EURion constellation

glossy stripe(s) that reveal additional information when viewed on anangle

watermark(s)

raised printing

ultraviolet ink (which reveals additional image information when viewedunder ultraviolet light)

thermochromatic ink (the color of which is dependent on temperature)

color changing or color shifting ink (the color changes depending onviewing angle)

see through indicia with complementary front and obverse portions

microprinting (high resolution printing)

unique printing fonts

embedded security threads

perforations

plastic, cotton fiber or other material matted or other texturedsurface(s)

barcodes

serial numbers

foil stamps

photographs including clear window photographs

unique metal compositions (for coins) such as bi-coloration other.

Example non-limiting recognition processes as described above canexclude such security features, or may take them into account or usethem as part of the recognition process. However, since the purpose ofthe recognition is not to authenticate the photographed item as beinggenuine, the recognition/matching algorithm is quite different from onesthat are used for banknote or ID authentication. In example non-limitingembodiments for example, it is desirable that matching occurs based onphotographing a copy (e.g., displayed on a tablet screen or the like)and not just an original of a banknote, ID or the like. Thus, thematching will achieve positive results based on counterfeit(inauthentic) banknotes or IDs. However, the matching/recognition isrobust in being able to detect different banknote denominations (e.g.,10 euro note versus 20 euro note versus $100 US bill etc.) and thusprovide different overlays depending on which banknote is recognized.

In still other embodiments, the two-dimensional or three-dimensionalobject could comprise a political campaign poster, billboard, flyerpackage, floor stand, table stand, pen with advertisement or otherprinted material or 3 dimensional object with print and the overlaycould provide a campaign song, slogan, speech, story, video or othercontent. Other types of campaigns could include advertising campaigns,company logos and printed advertising or any other printed campaign fromcompanies or other entities.

As discussed above, other embodiments can employ a two-dimensionalobject comprising a real estate advertisement, a job advertisement, aproduct advertisement, a service advertisement, or any otherpresentation of an item, entity or organization that a user may wish toobtain more information concerning.

In example embodiments herein, the two-dimensional object is not (anddoes not need to contain) a so-called “AR marker” or 2D bar code and isotherwise not specially designed or intended to be recognized by anaugmented reality display device.

Example 1

FIG. 5A shows a smart device 202 the camera of which has captured animage of a $20 bill. Once the app recognizes the captured image 402 asdescribed above, it displays a corresponding advertisement 404 as shownin FIG. 5B. The advertisement may be superimposed over the capturedimage 402.

Example 2

FIG. 6A shows a smart device 202 the camera of which has captured animage of a US passport. Once the app recognizes the captured image 406as described above, it displays a corresponding video 408 (as shown inFIG. 6B) that is superimposed onto the captured image 406. If the userrotates the smart device 202 to change the position and/or orientationof the captured image 406, the corresponding video 408 will also changeits position and/or orientation by the same amount. In some embodiments,the video 408 may be displayed with controls such as play/stop,fast-forward, rewind and mute that the user can press on the touchscreen display 210 to allow the user to control playback of the videoand/or accompanying audio.

Example 3

Instead of a video, FIG. 6B could display a superimposed control barthat allows the user to play an audio story associated with the capturedimage 406.

Example 4

FIG. 7A shows a captured image of a print advertisement. FIG. 7B showsan example area of recognition. FIG. 7C shows a superimposed virtualimage (shopping carts) on the FIG. 7A captured image to provide avirtual or mixed reality composite image.

Example 5

FIG. 8A shows an example of a captured image of a print advertisement.FIG. 9A shows an example area of recognition. FIG. 9B shows asuperimposed virtual image (a shopping cart) on the FIG. 8A capturedimage to provide a virtual or mixed reality composite image.

Example 6

FIG. 10A shows a captured image of a product package with a defined areaof recognition. FIG. 10B shows a superimposed virtual or mixed realitycomposite image providing a video of ingredients.

Example 7

FIG. 11A shows a captured image of a print advertisement for real estatelistings. FIG. 11B shows a closeup of an area of recognition. FIG. 11Cshows a superimposed virtual image on the FIG. 11B captured image.

FIG. 12 shows an example captured image of a human resourcessadvertisement. This captured image can have an area of recognition andcan have a digital image superimposed thereon to provide an augmented ormixed reality image on a smart device such as a smartphone, smartglasses, etc. In some embodiments, the display process may compriseprojecting the digital overlay directly or indirectly onto a user'sretina. Such techniques would allow the user to view the digital overlayon a screen (e.g., smartphone) and/or on glasses that are used as ascreen and/or independently of any screen viewing (i.e., the user couldsimply view the real world and the digital overlay would become part oftheir field of view based on such projection directly onto theirretina).

All printed publications cited above (including the ones shown in FIGS.2A-2J) are hereby incorporated by reference as if expressly set forth.

While the invention has been described in connection with what ispresently considered to be the most practical and preferred embodiments,it is to be understood that the invention is not to be limited to thedisclosed embodiments, but on the contrary, is intended to cover variousmodifications and equivalent arrangements included within the spirit andscope of the appended claims.

1. A portable display device for automatically recognizing two or threedimensional real world objects and augmenting or enhancing display ofsuch real world objects with an additional presentations superimposedthereon, the portable display device comprising: a portable housinghaving a surface, a touch screen display disposed on the housingsurface, a push button disposed on the housing, a camera disposed withinthe housing, a memory disposed within the housing, a wirelesscommunications device disposed within the housing, and a processordisposed within the housing and operatively coupled to the touch screendisplay, the memory, the camera, the push button and the wirelesscommunications device, the processor configured to perform, in responseto instructions stored in the memory, operations comprising: (a)display, on the touch screen display, an image of a real world objectacquired by the camera as the camera is aimed at the real world object;(b) recognize at least a portion of the acquired image without requiringa bar code or AR marker or other special marking that is not intended tobe recognized by humans but instead is designed or intended to beautomatically recognized electronically by a machine; and (c) inresponse to recognizing at least a portion of the acquired image,anchoring a presentation to the displayed image of the real world objectcurrently being acquired by the camera such that if the housing is movedto change the position and/or orientation of the housing relative to thereal world object, the presentation moves with apparent movement of theimage of the real world object on the touch screen display, wherein thepresentation provides offering details concerning the recognizedacquired image portion.
 2. The portable display device of claim 1wherein the processor is further configured to change position,orientation and/or perspective of the anchored presentation as theunderlying real world object changes position, orientation and/orviewing angle relative to the housing, in order to provide aphotorealistic image in which the anchored presentation appears to bepart of the underlying real world object.
 3. The portable display deviceof claim 1 further including at least one sensor disposed in thehousing, the sensor configured to sense posture, orientation and/orposition of the device, wherein the at least one sensor comprises aninertial sensor such as an accelerometer and/or a gyrosensor, andwherein the processor is configured to display, on the touch screendisplay, different portions of the presentation based on such sensedposture, orientation and/or position.
 4. The portable display device ofclaim 1 further comprising an optical flow sensor disposed within thehousing, the optical flow sensor providing an optical flow imageanalysis, and wherein the processor is configured to display, on thetouch screen display, different portions of the presentation based onsuch optical flow image analysis.
 5. The portable display device ofclaim 1 wherein the presentation comprises a mixed reality (MR) imagethat mixes virtual and real scenes.
 6. The portable display device ofclaim 1 wherein the presentation comprises an augmented reality (AR)image that augments real scenes with virtual objects.
 7. The portabledisplay device of claim 1 wherein the presentation is animated.
 8. Theportable display device of claim 1 wherein the presentation is displayedwith controls such as play/stop, fast-forward, rewind and mute that canbe pressed on the touch screen display to control the presentation. 9.The portable display device of claim 1 wherein the presentationcomprises a superimposed control bar displayed on the touch screendisplay that controls playing of the presentation.
 10. The portabledisplay device of claim 1 wherein the presentation comprises asuperimposed virtual shopping cart to provide a virtual or mixed realitycomposite image and the processor is further configured to wirelesstransmit an acceptance of the offering details in response to touchscreen manipulation of the superimposed virtual shopping cart.
 11. Theportable display device of claim 1 wherein the presentation is anchoredas if it were glued or otherwise adhered or attached to the real worldobject.
 12. A portable display device for automatically recognizing twoor three dimensional real world objects and augmenting or enhancingviews of such real world objects with an additional presentationsuperimposed thereon, the portable display device comprising: a display,a push button, a camera, a memory, a wireless communications device, anda processor operatively coupled to the display, the memory, the camera,the push button and the wireless communications device, the processorconfigured to perform, in response to instructions stored in the memory,operations comprising: (a) acquire an image with the camera; (b)recognize at least a portion of the acquired image without requiring abar code or AR marker or other special machine-readable marking; and (c)in response to recognizing at least a portion of the acquired image,anchoring a display of a presentation to a view of the real world objectsuch that if the display device is moved to change the position and/ororientation of the real world object relative to the display device, thepresentation moves on the display with movement of the view of the realworld object, wherein the presentation provides offering detailsconcerning the recognized acquired image portion.
 13. The portabledisplay device of claim 1 wherein the processor is further configured tochange position, orientation and/or perspective of the anchoredpresentation as the underlying real world object changes position,orientation and/or viewing angle relative to the display device, inorder to provide a view in which the anchored presentation appears to bepart of the underlying real world object.
 14. The portable displaydevice of claim 1 further including at least one sensor configured tosense posture, orientation and/or position of the device, wherein the atleast one sensor comprises an inertial sensor such as an accelerometerand/or a gyrosensor, and wherein the processor is configured to display,on the display, different portions of the presentation based on suchsensed posture, orientation and/or position.
 15. The portable displaydevice of claim 1 further comprising an optical flow sensor providing anoptical flow image analysis, and wherein the processor is configured todisplay, on the display, different portions of the presentation based onsuch optical flow image analysis.
 16. The portable display device ofclaim 1 wherein the presentation comprises a mixed reality (MR) imagethat mixes virtual and real scenes.
 17. The portable display device ofclaim 1 wherein the presentation comprises an augmented reality (AR)image that augments real scenes with virtual objects.
 18. The portabledisplay device of claim 1 wherein the presentation is animated.
 19. Theportable display device of claim 1 wherein the presentation is displayedwith controls such as play/stop, fast-forward, rewind and mute that canbe actuated to control the presentation.
 20. The portable display deviceof claim 1 wherein the presentation comprises a superimposed control barthat controls playing of audio.